One Step Closer to Automatic Evaluation of Text Simplification Systems
نویسندگان
چکیده
This study explores the possibility of replacing the costly and time-consuming human evaluation of the grammaticality and meaning preservation of the output of text simplification (TS) systems with some automatic measures. The focus is on six widely used machine translation (MT) evaluation metrics and their correlation with human judgements of grammaticality and meaning preservation in text snippets. As the results show a significant correlation between them, we go further and try to classify simplified sentences into: (1) those which are acceptable; (2) those which need minimal post-editing; and (3) those which should be discarded. The preliminary results, reported in this paper, are promising.
منابع مشابه
A survey on Automatic Text Summarization
Text summarization endeavors to produce a summary version of a text, while maintaining the original ideas. The textual content on the web, in particular, is growing at an exponential rate. The ability to decipher through such massive amount of data, in order to extract the useful information, is a major undertaking and requires an automatic mechanism to aid with the extant repository of informa...
متن کاملLexico-syntactic text simplification and compression with typed dependencies
We describe two systems for text simplification using typed dependency structures, one that performs lexical and syntactic simplification, and another that performs sentence compression optimised to satisfy global text constraints such as lexical density, the ratio of difficult words, and text length. We report a substantial evaluation that demonstrates the superiority of our systems, individua...
متن کاملAutomatic Text Simplification for Spanish: Comparative Evaluation of Various Simplification Strategies
In this paper, we explore statistical machine translation (SMT) approaches to automatic text simplification (ATS) for Spanish. First, we compare the performances of the standard phrase-based (PB) and hierarchical (HIERO) SMT models in this specific task. In both cases, we build two models, one using the TS corpus with “light” simplifications and the other using the TS corpus with “heavy” simpli...
متن کاملReadability Indices for Automatic Evaluation of Text Simplification Systems: A Feasibility Study for Spanish
This paper addresses the problem of automatic evaluation of text simplification systems for Spanish. We test whether already-existing readability formulae would be suitable for this task. We adapt three existing readability indices (two measuring lexical complexity and one measuring syntactic complexity) to be computed automatically, which are then applied to a corpus of original news texts and...
متن کاملTestuen sinplifikazio automatikoa: arloaren egungo egoera
The aim of this paper is to give an overview of the state-of-art in automatic text simplification. To that end, we present the systems and prototypes according to the language the are built for, their target audience and the type of simplification (syntactic, lexical or both) they perform. Moreover, we expound the di↵erent evaluation methods that have been carried out with these systems and the...
متن کامل